Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter_parser: fix reserve data and preserve key handling #9675

Merged
merged 1 commit into from
Dec 3, 2024

Conversation

niedbalski
Copy link
Collaborator

Handle properly the 4 combinations for Reserve_Data and Preserve_Key:

  • Off/Off: keep only parsed fields
  • Off/On: parsed fields + original key
  • On/Off: all fields except target key
  • On/On: keep all fields

Testing

  • Example configuration file for the change
[SERVICE]
    Parsers_File ./parser.conf

[INPUT]
    Name dummy
    Tag  dummy.data
    Dummy {"data":"100 0.5 true This is example", "key1":"value1", "key2":"value2"}

[FILTER]
    Name parser
    Match dummy.*
    Key_Name data
    Parser dummy_test
    Reserve_Data Off
    Preserve_Key On

[OUTPUT]
    Name stdout
    Match *

Parsers

[PARSER]
    Name dummy_test
    Format regex
    Regex ^(?<INT>[^ ]+) (?<FLOAT>[^ ]+) (?<BOOL>[^ ]+) (?<STRING>.+)$
  • Debug log output from testing the change

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Handle properly the 4 combinations for Reserve_Data and Preserve_Key:
- Off/Off: keep only parsed fields
- Off/On: parsed fields + original key
- On/Off: all fields except target key
- On/On: keep all fields

Signed-off-by: Jorge Niedbalski <[email protected]>
Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me, it looks good.

@niedbalski niedbalski merged commit 4ed325c into master Dec 3, 2024
48 of 50 checks passed
@niedbalski niedbalski deleted the fix/preserve-key branch December 3, 2024 11:01
AdheipSingh pushed a commit to AdheipSingh/fluent-bit that referenced this pull request Dec 4, 2024
Handle properly the 4 combinations for Reserve_Data and Preserve_Key:
- Off/Off: keep only parsed fields
- Off/On: parsed fields + original key
- On/Off: all fields except target key
- On/On: keep all fields

Signed-off-by: Jorge Niedbalski <[email protected]>
Co-authored-by: Jorge Niedbalski <[email protected]>
Signed-off-by: AdheipSingh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants